This separate section for wcag specs is very exciting.

• Would there be separate charts re: wcag 2.0 vs 2.1?
• General rule of thumb is automated tests can only cover ~30% of a11y tests so this can't be automatically updated after CI runs tests. It would be best if there was some sort of object of data associated with each component that these results could be culled from.

@scottnath, @dakahn and I are working through some of those details, but here's where we're at so far. Each component's spec could include 1 to many accessibility classifications. Each classification (e.g. WCAG 2.0 AA) has a subset of spec tests needed to hit that level of accessibility, plus a time-stamped summary of manual test results performed on a component that passes 100% of that subset of tests. That way, should we need to add test cases to that accessibility classification, or change anything, we know to re-run manual tests.

In this scenario, maybe the component's Specification column has two available accessibility badges (WCAG 2.0 A and WCAG 2.0 AA), and then each implementation has at most one accessibility badge (e.g. React component is passing all of the tests needed to be AA so it's AA, Angular component passes all the A tests but has not implemented the changes to pass the AA tests, so it's A).

I'll share an example which will be easier to understand.

spec numbering

confused on the numbers on these. What does a 1 ( ) mean here vs a 9 ( ) above?

data and inheritance

Thinking about where all this data will come from. For our components, we track component-level and demo-level data and store that data in a [component].config.js file (just like what's in each component in carbon-components). This structure allows the testing-data to be inherited by demos from their parent component.

Example (from our core) image component of how we track browser support at the component level:

const support = {
  browsers: {
    chrome: 'true',
    firefox: 'true',
    safari: 'true',
    edge: 'true',
    ie11: 'true',
  },
...

but then those can be overridden at the demo level:

const picture = {
  name: 'picture',
  label: 'Picture',
  support: {
    browsers: {
      ie11: {
        content:
          'The `picture` scenario does not meet all PAL browser requirements.' +
          ' This method can only be used for applications that do not need to support IE11.',
        url:
          'https://developer.mozilla.org/en-US/docs/Web/HTML/Element/picture#Browser_compatibility',
        support: 'mixed',
      },
    },
  },
};

demo wcag rating

How does the spec rating work vs the wcag rating? ie...if Accordion is rated at wcag-2.0-aa, then by passing all spec tests would a component and/or demo be considered aa? What if a given demo or component had a better rating of aaa?

I'm wondering if the wcag rating could be tracked as a separate data point at the data & component & framework level:

Button:
  Spec: WCAG-aa
  Vanilla: spec-passing (WCAG-aa)
  Vue: primary button @ WCAG-aaa, 
      component @ WCAG-aa means:
      vue-button/primary=wcag-aaa, but vue-button=wcag-aa, but _both_ passing spec

	Type	Vanilla	React	Angular	Vue
General	-
Primary	variant

I'm working on component configs (living in package.json files once each component is individually published) and revisiting component directory structure now that we have a better understanding of where we want to go compared to a couple months ago. Let's go over that next sync-up! Still a WIP over here.

Also confused on this one's meaning:

I'll document what those spec test result badges mean. In that scenario, 2 tests pass and there is 1 warning.

What would constitute a warning? Maybe a test that addresses something that takes the component from WCAG 2.0 AA to AAA. The component isn't failing - but it could be better.

What if a framework doesn't want to deprecate? Would the spec stay forever?

I think we could come up deprecation logic at each major release similar to what @joshblack outlined here: carbon-design-system/rfcs#5 (comment)

We'll have to clearly document community component governance so any directory of community components doesn't become a graveyard of unsupported components.

@scottnath, @dakahn and I are working through some of those details, but here's where we're at so far. Each component's spec could include 1 to many accessibility classifications. Each classification (e.g. WCAG 2.0 AA) has a subset of spec tests needed to hit that level of accessibility, plus a time-stamped summary of manual test results performed on a component that passes 100% of that subset of tests. That way, should we need to add test cases to that accessibility classification, or change anything, we know to re-run manual tests.

In this scenario, maybe the component's Specification column has two available accessibility badges (WCAG 2.0 A and WCAG 2.0 AA), and then each implementation has at most one accessibility badge (e.g. React component is passing all of the tests needed to be AA so it's AA, Angular component passes all the A tests but has not implemented the changes to pass the AA tests, so it's A).

I'll share an example which will be easier to understand.

I think we could come up deprecation logic at each major release similar to what @joshblack outlined here: carbon-design-system/rfcs#5 (comment)

We'll have to clearly document community component governance so any directory of community components doesn't become a graveyard of unsupported components.

I'll document what those spec test result badges mean. In that scenario, 2 tests pass and there is 1 warning.

What would constitute a warning? Maybe a test that addresses something that takes the component from WCAG 2.0 AA to AAA. The component isn't failing - but it could be better.

I'm working on component configs (living in package.json files once each component is individually published) and revisiting component directory structure now that we have a better understanding of where we want to go compared to a couple months ago. Let's go over that next sync-up! Still a WIP over here.